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Knowing and understanding the principles and tenets 
underlying dissemination, utilization? tind impact collection 
facilitate the "how." Dissemination is a communication process 
involving interactional relationships among individuals. Involvement 
of multiple agencies gives ri^eNio networking. Disseminating is also 
a major facet of planned change, one stage* of which is acquisition — a 
concern of the National Jletvork for Curriculum. Coordination in 
Vocational and Technical Education (NNCCVTE) • The concept of 
utilization demands that both product and process impact assessment 
be considered. Nine majcr prob^tems hinderthe acquisition of sound 
impact data; problems in te aggregating cfata, the myth of theory, 
inaccurate 6r incomplete specification of an instructional model, t 
ertor of measurement, inappropriateness of certain models, neglect of 
the .dissemination component by instructional materials developers, £y 
neg^ot of the communication systems model, variations in source 
credibility, and improper, funding for dissemination activities* 
NNCCVTE dissemination pe^sohnel ^concerned with impact ass'essmeat must 
focus on these questions: Are minrioal conditions present for impact 
to occur?, What factors are important in determination ox program 
consequence?, Are followup studies important?. What is the .place of 
experimental designs?. What are major purposes of impact assessment?, 
and What facts influence impact ,data in program planning Dased on 
manpower data? (ILB) 
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STATE DISSEMINATION PLANNING AS PART 
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James E. Wall 



When I was approached about preparing a presentation on dissemination, utilization and impact 
"collection7the as^nmenl given me stated that personnel of the National Network for Curriculum 
Coordination in Vocational and Technical Education (NI^CCVTE) have been much concerned about 
the various-states' capacities to disseminate R&D outcomes, particularly the curriGulum materials 
resulting from our vocational research efforts. Furthermore, the assignment given me indicated that: 
"We would like your remarks to address dissemination and utilization issues and particularly 
the issue of $pact collection " . 

The assignment further stated that NNCCVTE personnel are all aware of the imperatives of these two 
major priorities. Also, the assignment states fhat, to date, little had been shared with NNCCVTE 
personnel in regard to havv^and where to start in these areas. J was asked to share any findings which 
would be useful to the NNCCVTE group and which could be applied in a consistent manner. Finally, the 
assignment clearly indicated the NNCCVTE pe/sonnel were seeking ways which would permit th&» 
collection and aggregation of data across states in regard to dissemination, utilization and impact 
Collection, or D-U-IC for short. , •/* 

At the autset I n£ed to stress that this paper is more an adventure in the "why" than the "how" of 
dissemination, utilization and impact collection, or D-U-IC. Specifically, this paper will stress the 
principles and tenets underlying D-U-IC more than it will the particular dissemination and utilization 
steps and activities which usually always vary greatly in given situations because of differing administra- 
tive structures and management styles. The reason I do this is that we must recognize that we cannot 
just go around willy-n illy, moving an innovation or a curriculum material^ package from one place and 
putting it down in another, -without having sound insights and knowledge of the way in which the 
material interacted with its developmental environment, and how it is likely to interact with its new 
proposed environment, as well as how it is apt to interact with other related materials that already are in 
place in its proposed new home. In other words, it is not merely a matter of our playing out the role of a 
disinterested "technocratic" type of manager. It is a matter of knowing what vve are doing in dissemina- 
tion and how what w£ do fits into the overall efforts that are focused on vocational program improve- 
ment. It is a matter of knowing that what we do in dissemination takes a great degree of skill in a number 
of activities that draw upon competencies embodied in a variety of disciplines and sub-disciplines. 
What we do also takes time and fupds. And, just as importantly, what we do frequently demands that 
the SLRs have a thick skin, especially when our budget and manpower requests are ignored by our. 
administrative superiors and pur grant applications are turned down by decision-makers at the federal 
level. 

* t- .. 

My contention in this paper is that the "how" of D-U-IC can be better facilitated by knowing and 
understanding more about the "why." As a result, I will probably be making reference to concepts like 
the planned change process, change agents, networking, diffusion theory, curriculum development 
models, activity impact measures, perceived versus measured reasons accounting for successes, dis- 
semination models, communication models and many others. I may usetheseterms somewhat loosely 
artd me meanings \ attach to them may not coincide with the meanings found in the literature/ 
Nevertheless, the general purposes of this paper should become evident as the reader progresses. 



v Dissemination Overview 

Any dissemination effort or strategy is first and foremost a type of communication process. Dissemi- 
nation involves interactional relationships among individuals. In thecaseof the NNCCVTE, dissemina- 
tion requires that action be initiated from some kind of administrative^ organizational structure or 
structures, such as a local vc>cational program or a state vocational division or department. Or, the 
action may be initiated from the top down — the "administered" or "blueprint" approach — that is 
typified by strategies usejj in the National Diffusion Network. Structure, then, denotes multiple levels 
in some type of hierarchy. To add to the complexity of the dissemination process that is characterized 
by theSjNCCVTE, we must recognize the necessity for interactional relationships and communication 
between and among multiple agencies, which give rise to the concept of networking. For more insight 
into communication within and between organizations, the reader is referred to Communication in 
Organizations by Everett M. Roger* and Rekha Agarwala-Rogers, the Free Press (1976). 

Dissemination also is a major facet of the process of planned change. The prime mission of the 
NNCCVTE, at least as seen by this writer, is to aid vocational educators in acquiring resources (the 
concept of acquisition) for program improvement. NNCCVTE personnel, of necessity, must bechance 
agents skilled in change agentry which is a hefping relationship. For more insight into planned change 
and the place of acquisition as a stage in planned change, the reader is referred to The Change Agent '5 
Guide to Innovation in Education by Ropald G. Havelock, Educational Technology Publications (1973) 

Havelock describes six stages of planned change. (1) building a relationship; (2) diagnosis; (3) 
acquiring relevant resources, \4) choosing the solution, (5) gaining acceptance; and (6) stabilizing the 
innovation and generating self-renewal. To.be an expert change agent, one must become skilled in all 
six stages of th e planned change process. One m ust be able to see this process as it is applied in his/her 
organizational structure. If the planned change process does not exist in his/her organizational 
structure, then one must determine the best strategies- for helping to apply it. 

Since the NNCCVTE is most concerned* with acquisition, it seems wise to briefly describe in some, 
detail the acquisition stage of Havelock's model, although the remaining stages also are extremely 
important. Havelock reminds us that resource's come in many forms, such as print or graphic materials, 
people or products. In a helping relationship, one must Understand what has occurred — developmen- 
tal ly, what is available, and what is potentially>relevant and useful. He also indicates that acquisition is a 
three-part problem. diagnosis-awaren£S5*"homing" in on relevant resources. Techniques for acquir- 
ing diagnostic information are described in detail. Havelock presents awareness — ''what's out there" — 
as the key to an intelligent over.all acquisition strategy by describing how to build and maintain 
awareness of the f 'resource universe. 1 ' I would guess that this is the poinUt which the NNCCVTE makes 
its mpst vital contribution in the dissemination process. Finally, Havelock suggests a "homing-in" 
strategy in a six-step sequence: (1) overview from written source, (2) overviewjrom knowledgeable 
person, (3) observe "five" examples; (4) obtain evaluative data; (5) obtain innovation on trial, and (6) 
acquire a framework for evaluation-after-trial. Many of us perform this six-step sequence almost 
unconsciously. 

Impact Assessment 

With the above brief overview of dissemination and its relationships to communication and planned 
change, we turn now to impact assessment. For our purposes here, impact assessment must be defined 
and possibly confined. Perhaps we need to determine whether to focus on the measurement of impact 
of that (product) which is disseminated, or the impact of "in-place" activities (process) 1 of the NNCCVTE. 
The concept of utilization demands that we consider both, especially if we adhere to good systems 
theory'which demands features like "closed loops" and "feedback." # 

Before raising a lot of questions about product/process impact assessment, we need to look at some 
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of the major problems that hinder the>acquisition of sound impact data. A few of these problems are 
^briefly discussed here, soi£ie~ relate to product impact assessment and some relate to process impact 
assessment: • * ^ / 

1. Problems in aggregating data. Aggregation of data tends to cloudlhe picture sometimes instead of 
clarifying ft. For example, variables that express individual peculiarities may be missing, whether the 
unit of analysis is a person or aprogram. Individual peculiarities exist among students, among teacher* , 
and teacher behaviors, among programs and program components, and among instructional materials 
and their components. Problems of this nature rela% more to product than process impact assessment. 

2. The myth of theory. Most instructional materials, curriculum packages, or curriculum programs have 
been developed according to some instructional design theory or learning 'theory. It needs to be 
recognized that a theory is necessarily incomplete, an abstraction cannot explain everything. For 
example, during development a variable may have been inadvertently omitted that does affect an 
objective variable in the instructional materials. This omission would influence impact of the materials 
product. Similarly, omission of a key variable, such as ignonnj£advocacy-building, in the dissemination 
of the same instructional materials would influence the impact of the dissemination process. v 

Furthermore, it often seems that a theory becomes accepted and acceptable merely because of its 
place in a tektbook and its continued repetition and recognition by its proponents who lay claim to 
b£ing object iv,e-minded. Often, repetitious use of a theory is a foible of scholarship. For example, one, 
common and natural phenon^non is the repetition of theories or hypotheses once posited. In 
education, as in other fields, what begins as a very tentative guess often becomes by repetition an 
assumed fact and represents "the consensus of scholarly opinion." m „ 

3. Inaccurate or incomplete specification of an instructional model. Impact will be influenced if a given 
set of instructional materials or a curriculum package is notiased on a proven j(field tested, validated, 
widely recognized and accepted) development model, or if they are based on components from 
several models, or if based on only one component of a given model. ~~~ 

4. Error of measurement. Measures must betaken if impact is to be determined, and impact will be large- 
ly influenced by human behavior. Evenif human behavior were exact — and it isn't — measurement 
methodology is not, especially survey methods, opinionnaires, and the like. And, even experimental 
designs have many shortcomings when used rn impact studies, whereas in evaluation studiej such 
designs prove to be somewhat more acceptable. (This raises the question of differences between 
evaluation studies and impact studies, which may be partially explained in an oversimplified way by 
say ing that evaluation studies focus on measured changes in individuals as a result of being exposed to a 
curriculum, whereas impact studies focus on the changes in society and * longitudinal effects on 
individuals being exposed to a curriculum. The concept of causality rears its ugly head here.) 

5. Inappropriateness of certain models. Process models are the main consideration here. Scpne years ago 
when dissemination, diffusion, planned change»and innovation \#ere just beginning to be studied, the ( 
linear model was used to depict such processes. This was probably true because all communication 
models in vogue at the time also were linear. Linearity oversimplifies realityt A linear rt^odel does not 
include the concept of feedback. On the other hand,.a systems model does. The systems model denotes 
process, or cyclical actions, that is, a continuous sequence of actions through time. Depiciting dissemi- 
nation-utilization as a linear model does not portray the interactional complexity of the elements in the 
process. Linearity precludes the possibility of identify ing the forcing quality (the w£y in which the cause 
X acts on the result YJjhat is part of the inherent nature of X and Y variables. Dissemination research, 
and thus, impact assessment, probably should look to theoretical reasoning instead of empiricisrrUo 
determine why certain variables might have a forcing quality on others. Linearity demands that 
dependent and independent variables be identified, which is often futile in dissemination and com- 
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municatfon studies. An independent variable in a model can be changed or altered (source, message, 
channel, receiver); whereas the dependent variable consists of the measured effects of the action. The 
pertinent variables in a dissemination model may be interdependent; that is, each is a cause as well as a 
consequence of the others. Reality shows them to be a system of variables in mutual interaction^ 
'However, the reality-of dissemination models currently in operation is that they probably jend to be 
morelinear than systems oriented in character. One can readily see that this becomes a frustrating 
problem in designing impact studies. m % 

6. Instructional materials developers have, by and large, neglected the dissemination component. 

Curriculum developers or instructional materials developers are notoriously naive about andxisually 
misunderstand dissemination processes and problems. The result is that it is difficult to assess the 
impact of either the curriculum package or the dissemination process. Dissemination must be consi- 
dered (Turing development of materials if meaningful impact measures are to be obtained. 

7. Most dissemination models ignore tlje^mmunication systems model. Dissemination, like com- 
munication, is a multivariate, dynamic interplay of numerous elements. Impact study design must 
allow one to consider the over-time aspects of dissemination. Impact studies must go beyond'the 
immediate changes and include ths longitudinal effects of both product and process. 

A hypothetical situation seems useful at this point. Suppose, for example, the message variables are 
altered to measure the outcomes or effects onyreceivers of the message. That is, a similar message 
containing a fear appeal may be presented to a test group, but not to a control group. More specifically, 
one group of vocational instructors may be Jold that unless their output of competent trainees 
increases, their school will not continue to besupported and they will ultimately \oie their jobs. Another 
(control) group might be told of forthcoming increases in salaries and perquisites if similar production 
is raised. You can speculate what the responses to the two different messages might be. 

This introduces the concept of instructor incentives. Should instructors-receive extra compensation 
for being innovative, for installing competency based curriculum materials, fofr attending dissemina- 
tion conferences\vhere not only productsarepresented,butalsoprocessstagesandstrategies? If extra 
.compensation is initiated, how should it be handled? Could, for example, an instructor's salary be 
increased $5 per month if he/she attends a dissemination meeting on new and innovative materials, 
increased more if the materials are installed? (If this sounds outlandish, remember that most salary 
schedules include incremental increases for advanced degrees; why not for attending inservice 
sessions?kFin^ly, if such a strategy is used, how can it be evaluated and how can its long-range impact 
be assessed? For after all, if initiated it suddenly becomes a component of the dissemination model, and 
as such is subject to evaluation and/or impact assessment. 

8. Variation in source credibility. Oneof the most difficult problems to deal with in impact assessment is 
variation in source credibility. This is especially crucial in product impact assessment, but also is 
problemmatic in process impact assessment. Sources from which instructional materials emanate vary 
greatly in credibility. Hence, impact studies of materials must consider this factor when the study is 
being designed. Credibility has many dimensions, such as experience and prestige of the developing 
agency, the extent to which thesource has validated and field-tested the curriculum package prior to 
publication andVelease for dissemination, the reliability of the theoretical underpinnings of the 
instructional systems model that was followe3 during development, and others. 

9. Improper funding for dissemination activities. Apparently when funds are allocated for dissemina- 
tion, the decision-maker in his infinite wisdom compares dissemination of vocational curriculum 
.materials with public* relations activities. The most expensive part of any news or public-relations 
operation is the staff time (writer's time and source's time) spent on the initial copy. Replaying, 
repackaging and mailing the resulting materials is less than 20% of the total cost. This is not the way it is 
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for dissemination '^ince this step may cost many times what fhe developmental costs were, especially 
when inservicing costs are taken into consider alion. One surely should add costs of makirvg impact 
assessments o to the total, if not to the dissemination stage alone. 

Finally, most problems surrounding impact assessment fall into one orTnore of the following broad 
categories: 

A. Lack of a process orientation. » - ^ 

B. Ignoring or disregard of mutual causality among model elements in the dissemination process. 
C Focus on the individual (person or program) as the unit of-analysis instead of focusing on the 

intefretationships arid interdependence between persons involved in dissemination and concomitant"" 
communication processes. 

D Too frequently we have only a linear dissemination model in operation, but we keep trying to 
design our impact studies of the dissemination process along systems model lines. ' 

Summary 

NNCCVTE personnel have an obligation to know their clients, to help diagnose their needs, to 
establish communication in dialogue form or exchange, to become knowledgeable and aware of the 
"resource universe," to establish and maintain acquistion procedures and networks, and to engage in 
impact assessrnent. These are complicated undertakings, and yet they constitute only a few of the 
things that dissemination personnel are required to perform. 

In performing all of these things NNCCVTE personnel, especially SLRs, must continually ask them- 
selves the following kinds of questions: 

1. ARE MINIMAL CONDITIONS present for a particular change, innovation, or instructional package 
to take hold? Is the school environment conducive to "take" of the materials? Is the school ready ? What 
are the dimensions of being ready?. Are the instructors and other personnel prepared? Have the 
instructors, principals, administrators, counselors and other personnel formulated themselves into' 
self-supporting and self-renewing teams? 

2. ARE STUDENT COMPETENCE and placement important factors in impact assessment and determi- 
nation of program consequences? Are instruments, such 3s those from the National Assessment of 
Educational Progress (NAEP), available and accurate enough for assessing job knowledge and labor 
market skills, and what is their role in impact assessment? Are other benefits of vocational programs, 

- such as enhanced self-esteem, important to impact assessment? Row important are the following in 
impact assessment: (a) timing of the follow-up upon which the data are based, (b) method of data 
collection — from students or from teachers andtounselors, (c) training relatedness, and (d) economic 
conditions facing graduates as they enter jobs,' change fields, or relocate? 

3. ARE FOLLOW-UP STUDIES of program completers and leavers important to impact evaluation? Do 
systematic follow-up activities offer indications of impact of curriculums and programs on the post- 
program lives of participants, and do they offer possibilities for determining long-range impact on 
society and the economy? Can measures of preparticipation in a curriculum, such as background, 
preexistingdifferences, etc., becorrelated with postparticipation or outcome data foruse in enhancing 
impact assessment? How relevant are tfre methodologies and data of the NIE National Longitudinal 

/ t Surveys to impact assessment? 

4. DO EXPERIMENTAL DESIGNS have a place in impact assessment? Can component variables in the 
dissemination-»process be experimentally altered in an effort to determine such things as "forcing 
quality," i.e., which variables arepnost important in successful dissemination (the cause-effect issue)? 
5/ WHAT ARE THE MAJOR pOTposes of impact, assessment? 

A. To improve vocational education programs. 
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B. To satisfy accountability needs. 
C To fulfill requirements of legislation. 
How do these purposes influence impact study design? Should impact assessment be confined to 
\ major efforts conducted aUhe national level in longitudinal studies, or should impact assessment be 
^carried out at state and local levels? Does the previous question relate to problems in aggregating data? 
,Can management information systems currently being used contribute to impact assessment? Do 
^management information systems hold promise in forming the basis for longitudinal files that would 
contribute to impact assessment? 

©f INllrHE AREA OF PLANNJN<f of programs based on manpower data, are impact data unduly 
l influenced by-such facts as: many occupations (clerical, allied health, and service) have high turnover 
v characteristics because of abnormally low rates of pay, students continue to enroll in vocational 
.courses other than for vocational reasons, and projected net openings abused I to the exclusion of 
. \ot her data such as entry-level wages, wages at seniority, stu dent interests and propensity to relocate, job 
Satisfaction of occupational incumbents, employer and job entry requirements, programr costs, etc. 
- In summary, it seems tha^persons concerned with impact assessment must focus on determining 
what works, for whom, under what conditions, and all the possible outcomes. Persons conducting 
impact assessments must recognize: (1) that they are studying intervention strategies; (2) that these 
intervention strategies cauSe .intended and unintended results — some desirable and some not 
- — whjch result in a wide range of consequences to be assessed, (3) that any attribution of result to cause 
requires-sophisticated methodology which may or may not be readily available; ancL(4) that how 
outcomes occur or. fail to occur is needed to be known for makig£ decisions on both successful 
programs and those needing improvement. 

Although an exemplary impact assessment design is not contained in these pages, there is informa- 
tion her§ that should prove useful in designing and conducting impact assessments of dissemination 
processes. * 
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